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IMAGE PROCESSING SYSTEM FOR USE 
WITH A PATIENT POSITIONING DEVICE 

The present invention relates to image processing. More 
particularly, embodiments of the present invention relate 
to image processing systems for use with patient 
positioning devices. 

One application of the present invention is in the field 
of radiotherapy. Radiotherapy consists of projecting 
onto a predetermined region of a patient's body, a 
radiation beam so as to destroy or eliminate tumours 
existing therein. Such treatment is usually carried out 
periodically and repeatedly. At each medical 

intervention, the radiation source must be positioned 
with respect to the patient in order to irradiate the 
selected region with the highest possible accuracy to 
avoid radiating adjacent tissue on which radiation beams 
would be harmful. 

To this end, radiotherapy usually takes place in two 
stages- In an initial planning stage the patient's body 
is scanned using a 2-D x-ray simulator or a CT simulator, 
or by Magnetic Resonance Imaging (MRI) to visualise the 
target site and any obstacles. The course of therapy is 



then planned using the images obtained from the x-ray 
scanner or using the MRI images. Subsequently in a 
treatment stage a patient is irradiated in accordance 
with the course of treatment planned during the planning 
stage . 

A fundamental problem with radiotherapy is the need to 
position the patient in the same position, when obtaining 
diagnostic images and each of the subsequent times when 
radiation is applied to the patient's body. Present 
systems for positioning patients include various forms 
of systems for placing markers on the patient to enable 
the patient to be realigned for different applications 
of therapy. Thus for example US 5954647 discloses the 
use of a specially moulded bite plate on which LED's are 
placed to enable the head of an individual to be 
orientated in the same position in which diagnostic 
images are obtained. Similarly US 5446548 describes 
another positioning and monitoring system. The system 
of US 5446548 involves the fixing of infra-red reflective 
markers on parts of a patient's body. 

Although known systems involving the tracking of markers 
fixed relative to the patient enable the patient's 
positioning to be monitored, it is desirable to be able 



to determine the patient's position more accurately to 
increase the effectiveness of treatment* Further^ it is 
desirable to provide a positioning system which minimises 
the extra time required and the inconvenience to a 
patient arising from placing or fixing markers and thus 
reduces the stresses placed upon, the patient during 
treatment • 

In accordance with one aspect of the present invention 
there is provided a patient positioning system in which 
a three-dimensional model of the surface of a patient is 
generated utilising a number of stereoscopic video images 
of the patient. The video images are processed to 
generate the three dimensional model which is. rapidly 
updated to account for changes in the position of the 
patient . 

' In order to enable the surface model of a patient to be 
updated as rapidly as possible a nximber of techniques are 
utilised. In accordance with one aspect there is 
provided an image processing method comprising the steps 
of: 

obtaining stereoscopic images of an individual; 
generating a surface model of said individual 
utilising said stereoscopic images; 



receiving further stereoscopic images of said 
individual and generating a further model of said 
individual from said further stereoscopic images and said 
previous model of said individual. 

In accordance with a further aspect of the present 
invention there is provided a method of image processing 
comprising the steps of: 

obtaining a first and a second image of an 
individual from a first and a second view point; 

identifying within said first and said second 
images, portions of said images corresponding to the same 
points on the surface of said individual; and 

for said points on said surface of said individual, 
determining the orientation of the surface of said 
individual at said points relative to said view points, 
wherein said determination of said orientations relative 
to said view points comprises: 

determining the orientation of a point on the 
surface of said individual; 

for adjacent points on the surface of said 
individual for which said orientation has not been 
determined, determining whether said determined 
orientation for said point is a better estimate of the 
orientation of said adjacent points than a current 
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estimate of orientation for said adjacent points; and 

if said determined orientation is a better estimate 
than said current estimate, updating said current 
estimate for said adjacent points; and 
5 determining said orientation for said adjacent 

points utilising the best estimate of said orientation 
determined from said orientations for adjacent points. 

Further aspects and embodiments of the present invention 
10 will become apparent with reference to the following 

description and accompanying drawings in which: 

Figure 1 is a schematic diagram of a first 
embodiment of the present invention; 

Figure 2 is a perspective view of a camera rig of 

15 Figure 1; 

Figure 3 is a block diagram of the. computer of 

Figure 1; 

Figure 4. is a an overview flow diagram of the 
processing of the computer of Figure 1; 
20 Figure 5A is an illustration of a calibration device 

utilised to calibrate the positioning system of Figure 

1; 

Figure 5B is an exemplary illustration of an image 
obtained utilising the calibration device of Figure 5A? 
25 Figure 6 is a flow diagram of the processing of the 
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computer of Figure 1 for generating models of individuals 

from received images; 

Figures 7A and 7B are a flow diagram of the 

processing of the computer of Figure 1 for matching 
5 points in received images to determine the orientation 

of points on the surface of an individual; 

Figure 8 is a schematic illustration of a data 

structure for storing data utilised to perform the 

processing of Figures 7A and 7B; 
10 Figure 9 is a flow diagram of the processing of the 

computer of Figure 1 to generate a model of the surface 

of an individual; and 

Figure 10 is a schematic block diagram of a camera 

for use in a second embodiment of the present invention. 
15 First Embodiment 

Figure 1 is a schematic diagram of a first 

embodiment of the present invention. In accordance with 

this embodiment, there is provided a set of three camera 

rigs 1, 2, 3 that are connected by wiring 4 to a 
20 computer 5. The computer 5 is also connected to 

treatment apparatus 6 such as a linear accelerator for 

applying radiotherapy or an x-ray simulator for planning 

radiotherapy. A mechanical couch 7 is provided as part 

of the treatment apparatus 6 upon which a patient 8 lies 
25 during treatment. . The treatment apparatus 6 and the 



7 

mechanical couch 7 are arranged such that under the 
control of the computer 5 the relative positions of the 
mechanical couch 7 and the treatment apparatus 6 may be 
varied, laterally, vertically, longitudinally and 
5 rotationally. Also provided as part of the system is a 

laser projection apparatus 9 which is arranged to project 
three laser cross hairs 10 onto the body of a patient 
8 lying on the mechanical couch 7 where the three laser 
cross hairs 10 are such to identify the focussing point 
10 of the radiation generated by the treatment apparatus 6. 

In use, the cameras of the camera rigs 1, 2, 3 obtain 
video images of a patient 8 lying on the mechanical couch 
7. These video images are passed via the wiring 4 to the 

15 computer 5. The computer 5 then processes the images in 

accordance with the present invention to generate a 
three-dimensional model of the surface of the patient, 
which is displayed on the screen 11 of the computer 5. 
The three-dimensional model of the patient is then 

20 continually updated utilising the stream of video images 

obtained from the cameras of the camera rigs 1, 2, 3. 
The three-dimensional models of the surface of a patient 
8 are also utilised by the computer 5 to control the 
treatment apparatus 6 to position the mechanical couch 

25 7 relative to the treatment apparatus 6 in a consistent 
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manner throughout the course of a treatment. 

Prior to describing in detail the image processing 
algorithms utilised to generate and vary the three- 
5 dimensional model of the surface of a patient 8, the 

structure of the camera rigs 1, 2, 3 will, first be 
described in detail with reference to Figure 2. 

Figure 2 is a schematic illustration of one of the set 
10 of camera rigs 1, 2, 3 of this embodiment of the present 

invention. All of the camera rigs 1, 2, 3 in this 
embodiment are identical to the others with the three 
camera rigs 1, 2, 3 being arranged with one either side 
of the mechanical couch 7 and one at the head of the 
15 mechanical couch 7. The arrangement of cameras of the 

rigs 1, 2, 3 are such that all the cameras of the camera 
rigs view substantially the same portion of the patient 
immediately beneath the treatment apparatus 6 so that a 
complete model of that portion of the patient 8 can be 
20 generated. 

The camera rigs 1, 2, 3 each comprise a T-bar 12 which 
is suspended from the roof of the room within which the 
treatment apparatus 6 is provided. To this end a base 
25 plate is provided at the top of the T-bar 12 with the 
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main body of the T-bar 12 extending down from the centre 
of this base plate. Provided in each of the corners of 
the base plate 14 are fixing holes 15 through which bolts 
pass to fix the T-bar 12 in position in the room 
containing the treatment apparatus 6. 

Provided at the opposite end of the T-bar 12 to the base 
plate 14 attached to the horizontal cross bar 16 forming 
the T-section of the T-bar are three cameras 18,22,24. 
These cameras 18,22,24 are arranged along the cross bar 
16, and in order from left to right along the cross bar 
16 are a first geometry camera 18, a texture camera 20 
and a second geometry camera 22. The cameras 18, 20, 22 
each comprise monochrome analogue video cameras such as 
15 the Pulnix PEIOO. 



Provided in front of the lenses of each of the cameras 
is are filters 24, 26, 28. The filters on the two 
geometry cameras 18, 22 are arranged to prevent the 
geometry cameras 18,24 from deceiving light having a 
wavelength below 570 nm. The filter 26 in front of the 
lens of the texture camera 20 comprises a filter 
preventing the texture camera 20 from receiving light 
with a wavelength greater than 540 nm. 



20 
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Each of the cameras 18, 20, 22 is attached to the 
horizontal bar 16 of the T-l5ar 12 by a clamp 30 which 
attaches the cameras 18, 20, 22 to the horizontal cross 
bar 16 of the T-bar 12, whilst permitting the orientation 
5 of the cameras 18, 20, 22 relative to the horizontal bar 

16 of the T-bar 12 to be adjusted. 

Also attached to the horizontal bar 16 of the T-bar 12 
by further clamps 30 are a first 32 and second 34 light 

10 source. In this embodiment the first light 32 source 

comprises a light source arranged to irradiate light 
having a wavelength greater than 570 nm and being 
arranged to project onto the surface of a patient 8 lying 
on the mechanical couch 7 of the treatment apparatus 6 

15 a random speckle pattern. An example of a suitable light 

source would be a slide and slide projector, where the 
slide bears a non-repeating speckle pattern and the slide 
projector comprises a filtered light source filtered to 
prevent light with a wavelength less than 570 nm from 

20 being projected onto the slide. 

The second light source 34 comprises a conventional light 
bulb with a filter to allow light with a wavelength of 
less than 540 nm to pass. The second light source 34 is 
25 also arranged to illuminate the portion of a patient 8 
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which is visible from the cameras 18, 20, 22. The 
filtration of the light from the second light source 34 
prevents the light from the second light source 34, 
interfering and saturating the speckle pattern generated 
5 by the first light source 32. 

in use, images of the speckle pattern are obtained by the 
geometry cameras 18, 22 as these cameras are arranged 
to detect light having a frequency of greater than 570 

10 nm. These images from the geometry cameras 18, 22 are 

then subsequently processed so that the position and 
orientation of the surface of the body of the patient 8 
visible from the geometry cameras 18, 22 can be 
determined and a three dimensional model of the surface 

15 of the individual generated as will be described later. 

When a textured rendered model of a patient is desired, 
the images of the patient are obtained from the texture 
camera 20. These images are then utilised to texture 

20 render the model created from the geometry camera 18, 22 

images. As the texture camera 20 is arranged only to 
detect light having a wave length in less than 540 nm, 
this image does not contain the speckle pattern projected 
by the first light source 32 but rather corresponds to 

25 the visible images perceived by an operator within the 
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treatment room. 



Figure 3 is a schematic block diagram of the computer 5 
of Figure 1. The computer 5 comprises a microprocessor 
5 -38 and memory 40. The memory 40 is configured into a 

number of notional functional modules. 

In this embodiment, these functional modules comprise a 
processing module 42 for controlling the microprocessor 

10 38 to co-ordinate receiving and processing images; a 

control module 44 arranged to control the microprocessor 
38 to receive signals from the keyboard of the computer 
5 and co-ordinate the processing by the processing of the 
other modules; a calibration module 43 enabling 

15 microprocessor 38 to calibrate the positioning system 

so that relative to the point of focus of the treatment 
apparatus 6 as identified by the laser beam cross hairs 
10, the processing module 42 is able to identify the 
location of the surface of a patient 8; a mechanical 

20 couch control module .45 arranged to enable the 

microprocessor 38 to utilise generated surface models of 
an individual to generate positioning instructions which 
are then passed to the mechanical couch 7 of the 
treatment apparatus 6 so that a patient 8 may be 

25 automatically correctly positioned for treatment; an 
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image generation module 46 to enable the microprocessor 
38 to generate an image of the portion of the patient 
being treated and display it on the screen 11 of the 
computer 5; and a data store 47. 

5 

Also provided as part of the cpmputer 5 are a video sync 
unit 50 and a first and a second frame grabber 51, 52. 
The first and second frame grabbers 51, 52 are arranged 
to receive image data from the three cameras attached to 

10 each of the camera rigs 1, 2, 3, specifically the first 

and second frame grabbers 51, 52 are arranged to receive 
video signals from the geometry cameras 18, 22 of the 
camera rigs 1, 2, 3 via the video sync 50. Video images 
for the six geometry cameras (two cameras on each of the 

15 three camera rigs) are received by the video sync 50 and 

passed to the two frame grabbers 51, 52, three to the 
first frame grabber 51 and three to the second framb 
grabber 52. The video sync 50 passes timing signals from 
the first frame grabber 51 to the second frame grabber 

20 52 to ensure that image diata from the geometry cameras 

18, 22 is acquired simultaneously. By providing two 
frame grabbers 51, 52, in this way the computer 5 is 
arranged to receive image data from the geometry cameras 
18V 22 of all three rigs 1, 2, 3 simultaneously so that 

25 an instantaneous three dimensional model of the surface 



14 

of a patient 8 can be generated. 

Periodically, when requested by the control module 44, 
the video sync 50 is arranged to obtain image data from 
5 the texture cameras 20 of the camera rigs 1,2,3. When 

a frame of image data from these other cameras is 
received by the video sync 50 they are passed to the 
first frame grabber 51, and then onto the processing 
module 42. These images which comprise images generated 
10 from light having a wavelength less than 540 nm are then 

subsequently used to texture render the three dimensional 
model generated by the image generation module 46. In 
contrast to the geometry camera images received 
simultaneously for a particular frame, the texture camera 
15 images represent an images from the subsequent frame 

separated by approximately 40 milliseconds from the 
geometry camera images, utilised to generate a three 
dimensional model. However, as the appearance of the 
surface of an individual 8 varies very little over this 
20 time period, this mismatch in timing does not appreciably 

diminish the performance of the system. In contrast, 
simultaneous geometry camera images from all the geometry 
cameras are required to enable a highly accurate model 
of the surface of a patient 8 to be calculated. 



25 
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The overall processing of the computer 5 will now be 
described in detail. Figure 4 is a flow diagram of the 
processing of the computer 5 in accordance with this 
embodiment of the present invention. Initially a user 
5 via the keyboard of the computer 5 identifies to the 

control module 44 that the computer 5 is to process the 
images received from the three camera rigs 1, 2^ 3 to 
generate a set of calibration parameters to identify the 
relative positioning of the cameras 18, 20, 22 on each 
10 of the rigs 1, 2, 3. When such an instruction is 

received from the keyboard the calibration module 43 is 
invoked and camera calibration (SI) is then performed. 

A calibration sheet comprising a 40 x 40 cm sheet of flat 
15 rigid material such as aluminium or steel on which a 

pattern revealing a 20 x 20 matrix of circles at known 
positions on the surface of the sheet is provided. 
Additionally, towards the centre of the calibration sheet 
are four smaller markers adjacent to four circles the 
20 centres of which together identify the four corners of 

a square of known size. When calibration is to occur, 
the sheet is held in position on the mechanical couch 7. 
Images of the calibration sheet are then obtained by all 
of the cameras 18, 20, 22 of a camera rig 1, 2, 3 being 
25 calibrated. These images are then processed by the 
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calibration module 43 by initially performing a 
thresholding operation on the obtained images. The 
thresholded images are then processed to identify within 
the image the positions of the four markers in the images 
5 and their associated circles. This can be done either 

automatically using conventional techniques or 
alternatively, a user may identify the four circles 
manually. 

10 From the relative positions of circles identified by the 

markers in the images, a projective transformation is 
determined which accounts for the estimated centres of 
the identified circles defining the corners of a 
parallelogram in the image which arises due to the 

15 relative orientation of the calibration sheet and the 

camera obtaining the image. In this embodiment the 
transformation determined is an estimated transformation 
for distorting the image so that the circle centres 
correspond to the corners of a perfect square. 

20 

The calculated transform is then applied to each of the 
identified circles in turn to transform the oval shapes 
. of the circles. A more accurate estimate of the 
positions of the centres of the four circles is then 
25 determined by identifying the centre of the transformed 
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circles and utilising an inverse transform to determine 
the corresponding position of the estimated circle centre 
in the original image. These updated estimates are then 
revised to determine a new transformation. By repeating 
5 the process a number of times an accurate estimate of the 

transform required to account for the relative 
orientation of the calibration sheet can be made. 

Using the determined transform, the expected positions 

10 of all of the circles on the sheet appearing in the image 

are then calculated, the portions of the images in the 
vicinity of each of the estimated circle centres are then 
processed individually. Firstly, from the threshold 
image, the portion of an image corresponding to a circle 

15 on the calibration sheet is identified. The identified 

area is then subjected to the calculated transform so as 
to transform the elliptical appearance of the circle in 
an image due to the relative orientation of the 
calibration sheet to the camera being calibrated. The 

2 b actual centre of the transformed circle is then 

determined in a similar manner as has previously been 
described and then the projection of that actual centre 
appearing in the image is then calculated by applying the 
inverse of the calculated transform to the co-ordinates 

25 of the newly estimated circle centre. 
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When the co-ordinates for all the centres of each of the 
representations of the circles on the calibration sheet 
have been calculated for an image, the relative 
orientation of the different cameras 18, 20, 22 on one 
5 of the camera rigs 1, 2, 3 can then be calculated from 

the relative positions of these points in the images and 
the known relative locations of these circles on the 
surface of the calibration sheet as is described in 
detail in "A Versatile Camera Calibration Technique for 
10 High-Accuracy 3D Machine Vision Metrology Using Off the 

Shelf TV cameras and Lenses", Roger Tsai, IEEE Journal 
of Robotics and Automation, Vol. Ra-3, No. 4, August 1987. 
Further from the relative positions of the points in the 
individual images internal camera parameters such as the 
15 focal length, and radial distortion within the camera 

images can also be determined. This information is then 
stored within the data store 47 of the memory 40 of the 
computer 5 for use in subsequent generation of accurate 
three dimensional representations of the surface of a 
20 patient. 

When the camera positions and internal camera parameters 
have been determined for all of the cameras 18, 20, 22 
for all three of the camera rigs 1, 2, 3, the positioning 
25 of the cameras relative to the focussing point of the 
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treatment apparatus 6 is then detennined (32). 

In this embodiment as has previously been described a 
laser projection system 9 is provided which is arranged 
5 to project three cross hairs 10 identifying the focussing 

point of the treatment apparatus 6. This focussing point 
is also known as the isocentre for the treatment 
apparatus 6. In a conventional manner the calibration of 
this laser equipment and the treatment apparatus 6 is 

10 performed so that a user is able to use the projection 

of the laser cross hairs 10 being planes of light which 
intersect at the isocentre to identify the isocentre of 
the treatment apparatus 6 when treatment is occurring. 
The laser cross hairs 10 are then utilised to enable the 

15 relative positions of the cameras of the camera rigs 1, 

2, 3 to be determined relative to the isocentre. 

Specifically as is illustrated in Figure 5A, a 
ealibration cube comprising a cube of known size on whose 

20 surfaces are a series of thin black and white stripes is 

placed on the mechanical couch 7 in the vicinity of the 
focussing point of the treatment apparatus 6. As can be 
seen from Figure 5A the stripes on the calibration cube 
are oblique relative to the sides of the cube. The 

25 illumination within the treatment room is then switched 



off so that the cube is only illuminated by the cross 
hairs generated by the laser projection apparatus 9. 
Images of the cube illuminated in this manner are then 
obtained by the cameras 18, 20, 22 in each of the camera 
rigs 1, 2, 3. 

Figure 5B is an exemplary illustration of an image 
obtained by one of the cameras of the calibration cube 
illuminated only by the cross hairs generated by the 
laser projection apparatus 9. As can been seen in Figure 
5B the image comprises a number of discrete points 
corresponding to portions of the cross hairs which are 
reflected by the white stripes on the calibration cube. 

Processing pairs of images obtained by the geometry 
cameras 18, 22 from a camera rig 1, 2, 3 and the known 
relative orientation and internal parameters of the 
cameras 18, 22 on each rig, enables three dimensional co- 
ordinates of the points appearing in the images relative 
to the camera rigs i, 2, 3 to be determined. 

As these points correspond to points lying on planes 
intersecting with the surface of the calibration cube, 
groups of the points identify planes in space. The 
relative positioning of the cameras 18, 22, 24 and the 



focussing point of the treatment apparatus 6 can then be 
identified by determining the point of intersection of 
three planes which are identified by groups of points 
corresponding to pairs of the identified lines. 53 , 54; 
55^ 56; 57, 58 as this will be the only point in the 
treatment room where all the cross hairs intersect. Data 
identifying the co-ordinates of the isocentre relative 
to each of the camera rigs 1, 2, 3 is then stored in the 
data store 47 of memory 40 of the computer 5 so that the 
relative positioning of the cameras 18, 20, 22 of the 
three camera rigs 1, 2, 3 relative to this isocentre can 
be utilised as a fixed point to identify the relative 
position of a patient 8 on the mechanical couch 7. 

After the positions of all the cameras 18, 20, 22 of the 
camera rigs 1, 2, 3 have been determined relative to the 
focussing point of the treatment apparatus 6, the cameras 
18, 20, 22 can then be utilised to generate 
representations of the surface of a patient 8 lying on 
the mechanical couch 7. In order to do so an instruction 
is entered via the keyboard of the computer 5. When this 
is detected by the control module 44, the control module 
44 then causes the processing module 42 to start 
processing images received from the video cameras to 
generate a three dimensional model of the surface (S3) 
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received in those images. This processing continues 
until an instruction to stop (S4) generating images is 
received from the keyboard of the computer 5 at which 
processing of images and generation of surface models 
ends. 

The processing of the processing module 42 for generating 
models of the surface of a patient 8 on the mechanical 
couch 7 of the treatment apparatus 6 will now be 
described in detail with reference to Figures 6, 7A and 
B, 8 and 9. 



Figure 6 is a flow diagram of the processing by the 
computer 5 when a set of images are received by the 

15 processing module 42 from the frame grabbers 51, 52. 

Initially (SIO) the processing module 42 determines 
whether the images received are the first of a. set of 
images of the patient 8 which a model is being generated. 
If this is not the case, the processing module 41 will 

20 already have stored within the data store 47 data 

identifying the af fine transformations required to match 
points in images obtained by one of the geometry cameras 
18 of a camera rig in the corresponding image obtained 
by the second geometry camera 22 in the same camera rigs. 

25 If this is the case, the processing module 42 utilises 
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(Sll) the previously stored data in the data store 47 to 
initialise the model data being generated for the current 
images. The processing module 42 then (S12) proceeds to 
match point appearing in images obtained by the geometry 
cameras 18, 22 for the first of the camera rigs 1, 2 3 
as will now be described in detail with reference to 
figures 7A and 7B and 8. 



of 



Figures 7A and B are a flow diagram of the processing 
10 images by the processing module. 

As a first step in processing pairs of images received 
from the geometry cameras 18, 22 the processing module 
42 initialises (S20) the matrix of match parameters 
15 identifying the manner in which points in an image 

obtained from one of the geometry cameras 18 are matched 
in points in the image received from the other geometry 
camera 22. 

20 Figure 8 is a schematic illustration of a data structure 

for storing data utilised to match points in the images 
of one geometry camera 18 of a camera rig 1,2,3, to a 
corresponding point in images received from the other 
geometry camera 22 of the camera 1, 2, 3. The data 

25 structure of Figure 8 is stored in the data store 47 for 
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each of a matrix of points in the image of the first 
geometry camera 18 so that corresponding points in a 
corresponding image received from the other geometry 
camera 22 from the same camera rig can be identified. 

5 

For each of the points in the image which identify a 
matrix of patches in the image the following data is 
stored: an x coordinate 60 and a y coordinate 62 
identifying the location of the image patch being 

10 matched; a current transform 64 identifying the manner 

in which a patch in the first image is to be distorted 
so as to match the corresponding portion of a matching 
patch in a second image; an iteration result 66 
identifying the number of iterations utilised to 

15 calculate the current transform 64 or a negative code 

number identifying a lack of convergence; a patch 
difference 68 identifying the total difference in grey 
scale values, between the patch identified by the x and 
y coordinates 60,62 and the corresponding patch in the 

20 other geometry image; a process count 70 identifying the 

number of times the point identified by the x coordinate 
y coordinate data 60,62 has been, processed and a 
predict /calculated flag 72. 
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When the matrix of match parameters for a pair of images 
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is first generated for all of the pairs of x coordinates 
and y coordinates 60^62 the current transform 64 
iteration result 66, patch difference 68, process count 
70, and predict/calculated flag 7.2 are set as follows: 

Current Transform = identity 

Iteration Result = -99 

Patch Difference = 1000 

Process Count = 0 

Predict/Cal Flag = predicted 

If, however, a pair of geometry images from the camera 
rig 1/2,3 has already been processed, in this embodiment 
instead of initially setting the current transform to 
being the identity transform, the previously calculated 
transform for that point is utilised when initialising 
the matrix of match parameters for the next pair of 
images from that camera rig to be processed. In this 
way, the processing module 42 is arranged to assume that 
as an initial estimate of the surface of the patient, the 
shape of the patient 8 being monitored does not vary. 
As this is normally a reasonable first approximation, 
this ensures that the initial transformation selected as 
a starting point is to close to being correct, and hence 
the speed of which images can be processed is increased. 
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The processing module 42 then (S21) identifies the next 
patch in image received from a geometry camera 18 which 
is to be matched to a corresponding patch in a 
corresponding image from the other geometry camera 22 
5 from the same camera rig 1, 2, 3. This is achieved using 

a conventional search, in 3D space. 

Specifically, the image processing module, 42 first 
identifies the next patch in the image which has not been 

10 processed. This is achieved by scanning the matrix of 

match parameters to find an unmatched point. Using the 
co-ordinates of the selected patch and the relative 
orientations of the texture cameras 18, 22 on a camera 
rig, the image processing module then selects potential 

15 matches for the patch in one texture image as 

corresponding to the patch from the other image by 
calculating expected positions for the patch assuming the 
patch is a various depths away from. the cameras. When a 
match for the next patch has been identified, data 

20 identifying the coordinates of this patch and a potential 

match are added to the bottom of the queue (S22) of 
patches to be processed. 
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The image processing module 42 then (S23) selects the 
patch at the head of the queue for processing. The 
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transformation necessary to match the selected patch with 
a corresponding patch in the other geometry image being 
processed is then determined. 

5 In this embodiment assuming that the images, image 1 and 

image 2 being processed are represented by functions f 
and g such that: 

Image 1 = f (x,y) 
10 Image 2 = g (x,y) 

In a region being matched there will exist a transform 
T such that f(Xi,yi)=g(T(Xi.yi)), where Xi,yi are in region 
being matched. 

15 

Assuming that the transform T represents an affine 
transformation, which is a reasonable approximation the 
transformation T(x,y) can be represented as: 

20 T(x,y)=(ax+by+c,dx+ey+f ) 

Therefore in the regions which correspond to one another 
f (Xi,yi)=g(aXi+byi+c,dXi+eyi+f )+6 where 6; is an error term 
which reflects noise between the images. 
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Values for a,b,c,d,e and f can then be determined 
iteratively by relacing these values by a+5a, b+5b, 
c+5c,d+5d,e+5d,e+5e,f+5f and approximating the resulting ^ 
non linear function by a linear expression that can be 
5 solved using a least squares matrix technique. 

Thus, if gx(Xi',yi') and gyi^s.' rYi' ) are the derivatives of 

function g in the x and y directions evaluated at the 

point (Xi,yi) then from the above 

10 f (Xi,yi)-g(Xi' ,yi' ) = 

(xgx^5a+ygx^5b+g,^5c+xg/5d+ygy^5e+gyi5f ) +e with 

(Xi' ,yi' )=(axi+byi+c,dx+eyi+f) and 
gx ( Xi ' , yi ' ) =g^ , gy ( X ' 1 , yi ' ) =gy^ 
and hence in matrix form 
15 f(Xi, yi)-g(Xi',yi')=(xgxSygx^'gx^,xgyS ygySgyM 

*(5a,5b,5c,5d,5e,5f )''+6 

From the above the difference between the two patches 
f(Xi,yi) and g(x'i.y'i) is equal to the matrix 

20 multiplication of the two matrices on the right hand side 

plus the error Ex; and hence B=A5D+E where B is the 
difference between the two regions whose ith entry is 
f (Xi,yi)-g(Xi',yi'), D=(a,b,c,d,e,f ) , 6D=(5a,6b,5c,5e, 5f ) 
and A is a matrix whose ith row is 

25 (xgxSygxSgxSxgySygySgy^) which is dependent upon image 



derivatives and E is an error vector whose ith entry is 

Assuming UE)=0 and E,{BE'^)=o^-P'-^ where ^ is the 
expectation operator and a is a numerical factor and P is 
a matrix B=A 5D+E forms a Gauss-Markov estimation model , 
which can be solved by pre-multipiying by A^P so that 
A^B=A^PA5D+A^PE . Dropping the error term these can be 
solved by Cholesky Decomposition giving an unbiased, 
minimum variance estimator of 5D. 

An iterative determination of the values for a^b^c^d^e 
and f can therefore be made. With each iteration A should 
be updated to account for the new calculated transform T 
for the new values of a,b,c,d,e, f- This can however be 
more rapidly achieved by estimating the updated values 
for A for the next iteration from the previous values for 
A,B and D above as follows. 

From the above for points within portions of the images 
f (x^y) and g(x,y) being matched 

B=A5D+E 

where 5D=(6A,5b,5c,6d,5e,5f )^ and A is a matrix whose ith 



entry is (xg*,, yg^xr xg\, yg^y, g^) - 

Noting that A is equal, to the derivative of g with 
respect to the parameters (a,b,c,d/e,f), if after j 
iterations Bj,gj,D^ and are estimated values for B,g,D 
and A and assuming the patch in the first image f (x,y) 
stays fixed then after one iteration it is possible to 
determine 

5D=Di-Do and 

5g=gi-go = ( f i-gi ) - ( f o-go ) 
^Bj— Bo 

which gives infoirmation about the derivative of g in the 
direction 5D. Noting that, if 

A 6D = 5g, 

the derivative of g will satisfy the information 
obtained, it follows that an update of the iterative 
solution of A will be a reasonable solution where the 
above holds true. This can automatically be achieved by 
using the update. 

Aj=Aj_i+ ( 5gj-Aj_i5D3 ) ( 50^'' ) / ( 60^^60^ ) 
where 5Dj=Dj-Dj_i and 5gj=Bj_i-Bj; 

and hence by calculating for patches being matched an 
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initial difference matrix Bo and an initial derivative 
matrix 2^, an iterative solution for the transform T can 
be calculated solely by recalculating the difference 
matrix B, for each iteration and updating the transform 
5 T and the derivative matrix using this recalculated 

difference matrix Bj in the manner described with this 
update being a reasonable approximation. 

In this embodiment, the upper limit for the number of 
10 iterations is set to 10 and a patch is identified as 

having converged if the updated transform varies the x, 
y co-ordinates being matched by less than 0.03 pixels. 
In addition to checking whether convergence has been 
achieved, at each iteration in this embodiment a check is 
15 also made as to whether the calculated transform has 

failed either by introducing excessive expansion of an 
area, or varying the x, y co-ordinate of a potential 
match by 2 or more pixels or where a transform results in 
searching for a match outside the image area. If any of 
20 these conditions are met, this indicates that no 

realistic match will be reached. 

When convergence is reached data identifying the number 
of iterations used to reach a result is stored. 
25 Alternatively if the transform fails, a negative number 
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identifying that one of the errors has occurred is 
stored for the point as an iteration result. Otherwise, 
a new iterative solution for the transform is then 
determined. 

5 

Thus, when a patch is processed (S23) an iterative 
determination of the transform of the patch necessary to 
match the patch form image received from one of the 
geometry cameras 18 of a camera rig 1, 2, 3 with a 

10 corresponding patch in the image received from the other 

geometry camera 22 of the same camera rig 1,2 3 is made. 
Data for that point is then stored in the data store 47 
where the iterative transform is stored as the current 
transform 64 the number of iterations utilised to 

15 determine the result or a negative error number is stored 

as the iteration result 66 a patch difference value being 
the absolute grey scale difference for the patch 
transformed in the manner identified by the current 
transform 64 compared with the corresponding patch in the 

20 image received from the other geometry camera 22 the 

process count 70 is incremented by one and the 
predicted/calculated flag 72 is set to calculated. 



25 



The processing module 42 then (S24) determines whether 
the match made for the point being processed is a good 
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match, in this embodiment this is achieved by utilising 
the iteration result data 66 of - the patch. If, for any 
reason the number of iterations required to determine the 
transform 64 for matching the patch from one invention to 
5 the corresponding patch in the second image is too high 

or negative this indicates that it was not possible to 
determine a match for the point. If, however, this is 
not the case the processing module 42 utilises the 
transform 64 for the current point to act as seed data 
10 for initiating matches for adjacent points as will now be 

described in detail with reference to Figure 7B. 

Initially (S25) the processing module selects one of the 
four adjacent points adjacent to the point corresponding 

15 to the patch which has just been processed. That is to 

say the points at which the x coordinates and y 
coordinates 60, 62 stored in the data store 47 are one 
greater and one less than the patch which the x 
coordinates and y coordinates 62, 62 for the patch which 

20 has just been processed. 

When the adjacent point has been selected the processing 
module 42 then (S26) determines whether the process count 
70 for that point is less than three. If this is the 
25 case this indicates that the adjacent point which has 
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been selected has not previously been processed more than 
three times. If the point has been processed three 
times, this indicates that there have been difficulties 
in generating a satisfactory match for the point. In 
5 order to speed the processing of images, in this 

embodiment under these circxamstances the point is not 
considered for any further attempts at matching the 
point. In other embodiments the process count could be 
required to be a different value with the result more or 
10 fewer attempts at identifying a match for a point occur. 

If the process count for a point is less than three, the 
processing module 42 then determines (S27) whether the 
selected point is associated with a negative iteration 

15 result 66. This could arise either because the point has 

never been processed before or because the processing 
■ module 42 failed to deteannine a match for the patch last 
time the patch was processed. If the iteration result 66 
is negative, the processing module 42 sets the 

20 predicted/calculated flag 72 for the point being 

considered to be predicted and adds (S28) the point to 
the bottom of the queue. 

Thus in this way, the processing module 42 causes a 
25 queue of points to be identified which enable portions of 
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the images received from the geometry cameras to be 
matched in a manner in which adjacent portions of an 
image are processed consecutively. The adding of points 
to queue in this manner also enables an ordering of 
5 points to be made which requires minimal processing power 

and as will be described in detail later, enables the 
best matches for adjacent points to be used as seed 
values for subsequent processing. 

10 If the processing module 42 determines (S27) that the 

iteration result for a point is not negative, the 
processing module 42 then determines (S29) whether the 
adjacent point being tested has a predict/calculated flag 
set to predicted. If the predicted/calculated flag is 

15 set to calculated, this indicates that a transform for 

the point in question has already successfully been 
determined. However, if the predicated/calculated flag 
72 is set to predicted, this indicates that the transform 
for the point identified by the current transform data 64 

20 is only an initial estimate of a starting point for the 

transform necessary to match the patch identified by the 
X coordinate and y coordinate 62 of the selected adjacent 
point or when the point was processed previously, the 
processing module 42 failed to determine a satisfactory 

25 match for the point. 
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If the predict/calculated flag 72 is set to predicted the 
processing module 42 then (S30) compares the patch 
difference data 68 for the selected adjacent point with 
the calculated patch difference applying the current 
transform 64 of the previously processed point to the 
patch identified by the adjacent point being tested. If 
the application of the transform for the previous point 
which has been processed results in a better match 
between patches in the geometry images, this is indicated 
by the stored patch difference 68 for this adjacent point 
being greater than the calculated patch difference 
determined by applying the transform calculated for the 
previously processed point to the patch of the adjacent 
point being considered. If this is the case or 
alternatively automatically after adding a point 
associated with a negative iteration value to the queue 
of points to be processed (S28) the processing module 42 
(S31) updates the patch difference data 68 and the 
current transform 64 for the adjacent point being 
considered by setting the current transform 64 for the 
adjacent point equal to the transform for the previously 
processed point and the patch difference 68 for the 
adjacent point to the calculated patch difference 
calculated utilising the transform for the previous 
point . 
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After either the processing module has determined (S29) 
that the calculated flag 72 of the point is set to 
calculated, or the processing module 42 has deteirmined 

(530) that the transform for the previously processed 
5 point is not an improvement on the match for the current 

transform 64 for the adjacent point being considered or 

(531) the data for the adjacent point has been updated, 
the processing module 42 then (S32) determines whether 
all of the points adjacent to the point which has just 

10 been processed have been considered and updated if 

necessary. If this is not the case the next adjacent 
point is then considered (S25-S31). Thus in this way all 
of the four points adjacent to a point which is processed 
are considered one after another and the current 

15 transform data 64 for those adjacent points which have 

not already had satisfactory transforms calculated for 
them are updated to correspond to better initial 
estimates of transfoanns for those points. 

20 Returning to Figure 7 A after current transform data 64 

for adjacent points has been updated the processing 
module 42 then (833) determines whether the queue of 
points to be processed is empty. If this is not the case 
the next point in the queue is selected (S23) and a 

25 transform for that next point is calculated and estimates 
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of transforms are of adjacent points are updated where 
required (S24-S32). 

If the queue of points is empty the processing module 42 
then determines whether all of the points in the image 
responding to patches have been processed (S34) if this 
is not the case a new seed point (S21) is identified and 
further points are processed (S22-S33). When all of the 
points in the image have been processed (S34.) the 
processing of the processing module comes to an end. 

As an initial first approximation, it is reasonable to 
assume that adjacent parts of a patients body 8 are flat 
and orientated in the same direction relative to the 
cameras of the camera rigs 1, 2, 3. By processing 
patches in the images received from the geometry cameras 
18, 22 in the way described above, when the orientation 
of one point within the image has been determined, this 
orientation is utilised in selecting an initial start 
point for calculating the orientation of adjacent points. 
As the best orientation of either an identity transform, 
the orientation for a point determined from a previous 
pair of images, or the orientation of adjacent processed 
points is used as a start point for determining the 
transform, the intensive processing necessary for the 
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iterative accurate deteirmination of the orientation of 
points is minimised. The speed at which the processing 
module 42 is able to determine the orientation of the 
points appearing in the entire image is thereby 
increased. 

Further, by adding a point to the queue of points for 
processing when, an adjacent point is processed and the 
iteration result 66 for the point being considered 
indicates that no satisfactory match has been determined, 
the processing module 42 ensures that all available seed 
data is utilised to attempt to find a match for each 
point. Specifically, if an initial attempt at a match 
fails, a point is tested again by using any subsequent 
seed data determined for any adjacent points. Further 
although no attempt is made to identify the highest 
quality matches and process those matches first, the 
present embodiment ensures that by the time a point is 
processed all available information for any adjacent 
points is taken into account when selecting an initial 
stating transform. The adding of points to a queue of 
points to be processed also ensures that adjacent areas 
are processed together. 

Returning to Figure 6 when all the points in a pair of 
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images received from the two geometry cameras 18, 22 of 
the camera rig 1, 2, 3 have been processed, the 
processing module 47 then (S14) determines whether images 
for all three camera rigs 1, 2, 3 have been matched. If 
this is not the case the processing module 47 then 
selects the next set of geometry cameras 18, 22 (S15) and 
proceeds (S10-S14) to determine transformations to match 
portions of images received by the geometry cameras 18, 
22 of that camera rig 1, 2, 3. 



When transformations have been determined for all three 
pairs of geometry jLmages for a frame the processing 
module 42 then (S16) proceeds to generate a three 
dimensional wire mesh model of the surface of a patient 
15 8 utilising the calculated matches. 

Specifically as is shown in detail of the flow diagram of 
Figure 9, the processing module initially (S40) utilises 
the current transform data 64 associated with points 

20 identified by the x coordinates 60 and the y coordinates 

62 of data stored within the data store 47 for each pair 
of images together with the stored data identifying the 
relative orientations of the geometry cameras 18, 22 of 
the camera rigs 1, 2, 3 and intrinsic properties of those 

25 cameras 18,22 to generate three three dimensional wire 
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mesh models of the surface of a patient 8 as visible from 
the three camera rigs 1, 2, 3. In this embodiment the 
calculation of the wire mesh models of the surfaces as 
viewed from the three camera rigs 1, 2, 3 utilising this 
5 data is entirely conventional. 

The processing model 42 then selects a first model for 
processing. The processing module (S41) then identifies 
a triangle from the selected model defined by points at 

10 the edge of the selected wire mesh model. The processing 

module then determines for the selected triangle the 
projection of the selected triangle into the image planes 
of the geometry cameras utilised to generate, the other 
two models. The processing module then (S42) determines 

15 whether the selected triangle covered by a portion of the 

models generated from the other two camera rigs. 

Specifically the processing module determines for each of 
the projections of the vertices of the selected triangle 

20 into the view points of the other camera rigs, whether 

projections are also represented by the model generated 
using image data from the camera rig represented by that 
image plane. If this is the case for all three of the 
projected vertices into a particular image plane, the 

25 processing module 42 then determines whether the 
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triangles from the model generated utilising that image 
plane correspond to triangles in space which are 
relatively close to the triangle in space represented by 
the model being processed. This will be the case where 
5 the projected triangle is in fact representative of the 

same portion of the surfaces of a patient as part of the 
model generated utilising that particular image plane. 

If this is the case, the processing module 42 then (S43) 
10 determines whether the triangle being processed should be 

deleted. In this embodiment, a twofold test is utilised 
to determine whether a particular triangle should be 
deleted. The processing module 42 initially determines 
the relative orientation of the triangle being processed 
15 relative to the image plane utilised to generate that 

triangle. An angle identifying this relative orientation 
is then stored. The processing module 42 then makes a 
similar determination for the relative orientations of 
the triangles identified by the points of the vertices 
20 projected into the image plane used for the other models 

and for the triangles representing those vertices in the 
other models. 



25 



The triangle from the mesh being processed is then 
deleted if either the triangle from the mesh being 
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processed is determined to have an orientation greater 
than a current threshold orientation or the triangle is 
determined to be more oblique relative to the image 
planes utilised to generate the selected triangle than 
5 any of the orientations of the triangles from the other 

models identified by the projection of the selected 
triangle into the image plane utilised to generate those 
other models. In this embodiment the threshold value for 
deleting triangles is initially set at 90°. That is to 
10 say the processing module initially is arranged to delete 

triangles oriented at more than 90<» to or away from the 
image plane utilised to generate a portion of the model. 

If a triangle is determined to be completely represented 
15 by part of one of the other models and is determined to 

have been generated from data which is indicative of a 
more oblique angle than the threshold value or more 
oblique data than the data utilised to generate 
corresponding portions of the other models the processing 
20 module 42 then (S44) deletes that triangle from the 

stored model for the selected mesh. 

After either triangle has been deleted or if it has been 
determined that a boundary triangle being considered is 
25 not represented by a portion of the other models or is 
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not determined to have been generated utilising data 
indicative of a more oblique angle than the other models, 
the processing module 42 then (S45) determines whether 
all of the triangles at the edge of the mesh being 
5 processed have been considered for possible deletion. 

If this is not the case, the next boundary triangle in 
the currently selected mesh is selected and a 
determination whether to delete that triangle is made 

10 (S41-S44) when all of the triangles at the boundary of 

one mesh have been considered the processing module 42 
then (S46) determines whether all three of the wire mesh 
models have been considered. If this is not the case the 
next mesh is selected (S47) and then each of the boundary 

15 triangles at the edge of the wire mesh model represented 

by that mesh are individually considered for possible 
deletion (S41-S46). 

Thus in this way each of the triangles at the edge of all 
20 three meshes are considered and where the portion of a 

surface represented by the boundary triangle is 
determined to also be represented by other data, the 
triangle is deleted if it has been generated from more 
oblique data than the other data representing that 
25 portion of the surface. 
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After all three of the models have been considered in 
turn the processing module then (S48) determines whether 
in the latest round of processing any triangles have been 
deleted from the wire mesh models. 

5 

If this is the case the processing module 42 then (S49) 
determines whether the current threshold value for 
deleting triangles is set to its maximum level. In this 
embodiment the maximum level is set to 75°. If the 
10 current threshold value is not set at 75° the processing 

module 42 then (S50) reduces the threshold by a further 
5° before proceeding to select the first model for 
processing utilising the reduced threshold (S47). 

15 Thus in this way the criteria for deleting triangles from 

the various models are gradually reduced so that 
initially only the more oblique triangles which are 
represented by more than one wire mesh are removed. As 
the boundary triangles are processed at each iteration, 

20 the deletion of triangles will result in additional 

triangles being considered for deletion. This together 
with the relaxation of the deletion threshold causes the 
processing module to gradually reduce the extent of 
overlap of the surfaces represented by the three wire 

25 mesh models. 
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If after all of the meshes have been processed utilising 
a particular threshold, the processing module 42 
determines (S48) that no deletions have occurred in the 
latest round of processing or alternatively if the 

5 threshold value for deleting triangles has reached its 

maximum value, the processed models will comprise models 
where only a very small portion of the surface at the 
edge of one particular model is also represented by data 
from the other two models. The processing module 42 then 
10 (S51) proceeds to merge these minimally overlapping wire 

mesh models in the manner described in "zippered polygon 
meshes from range images" Greg Turk and Marc Levoy, 
Computer Science Department, Stamford University. When 
all three models have been processed and merged into a 

15 single model, the combined 3D wire mesh model is then 

utilised to generate positioning instructions to position 
the patient 8. 

Specifically returning to Figure 6 when a model has been 
20 generated by the image generation module 46 the control 

module 44 then (S17) determines whether the instructions 
received from the keyboard of the computer 5 indicate 
that the three dimensional wire mesh model data generated 
by the processing module. 42 is to be utilised to position 
25 the patient automatically by sending instructions to the 
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mechanical couch 7. If this is the case the control 
module 44 passes data for the generated wire mesh model 
together with data for a stored model from an earlier 
treatment session retrieved from the data store 47 to the 
mechanical couch control module 45. The mechanical couch 
control module 45 then performs (S18) the automated 
generation of movement commands for the mechanical couch 
7 in a two stage process. 



initially, the treatment couch control module 45 
determines a first rough transformation for aligning the 
surface identified by the newly generated wire mesh model 
with the surface identified by the stored model. 
Specifically, the treatment control module 45 selects a 
15 subset of the vertices from the generated model together 

with a decimated wire mesh representation of the stored 
model. This decimated model is determined utilising 
conventional techniques such as are described in 
'Decimation of Triangle Meshes' W. Schroeder et al 
20 Computer Graphics (SIGGRAPH '92) 26(2) pp65-70 August 

1992. 
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For each sample vertex in the generated model, the 
closest vertex to the other decimated stored model is 
determined. The closest point on the surface defined by 



48 

the decimated model for the stored surface to each of the 
sample vertexes to the generated model is then calculated 
by considering each of the triangles including the 
identified closest vertex in the decimated model. When 
5 the closest points from the sample vertices to the 

decimated surface have been determined an initial 
transform to align the generated surface to the decimated 
surface is calculated utilising conventional Procrustes 
algorithm techniques such as are described in 'Least 

10 Square Fitting of Two 3D Point Sets' Arun et al, IEEE 

Transactions on Pattern Analysis and Machine Intelligence 
vol PAMI-9 NO. 5 pp69«-700, 1987, and 'Closed Form 
Solution of Absolute Orientation Using Unit Quaternions' 
B.K.P. Horn J. Opt-Soc. Am.A. Vol 4, No. 4, pp629-642, 

15 April 1987. The process then iterates utilising the 

iterated closest point (ICP) algorithm as is described 
in "A Method for Registration of 3-D Shapes" Paul J. 
Besl and Neil D McKay, IEEE Transactions on pattern 
Analysis and Machine intelligence. Vol 14, No. 2 February 

20 1992 pp239-256. 

When determining the initial transform in this way, in 
order to identify the closest vertex from one model to 
that in another, a comparison of the distance between all 
25 of the vertices in second model must be made. The time 
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necessary for such a determination is dependent upon the 
number of vertices the each model being searched. Thus 
by decimating the stored model using conventional 
techniques the numbers of vertices to be searched is 
5 reduced and hence the time taken for registration 

minimised. 

After an initial transformation has been determined, the 
rough transformation is then refined by determining, for 

10 a sub-sampled selection of the vertices of the surface of 

the generated model to which the initial transformation 
has been applied, the projection of these vertices into 
the camera views utilised to generate the stored model. 
The closest vertex from the generated model to each of 

15 the vertices in the stored model is then determined by 

considering for each vertex in the generated model the 
projection of the vertex to the camera frames utilised to 
generate the stored model. The closest vertex being used 
in the stored model is then calculated using these 

20 determined closest pixels for each vertex. Having done 

this the ICP algorithm is repeated as described above to 
determine a final matching position. 

The mechanical couch control module 45 then transmits to 
25 the mechanised couch 7 instructions to orientate the 
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couch 7 in order to account for the determined 
differences between the prestored model and the current 
generated model. 

5 After any necessary movement commands have been 

generated,, the models are then passed to the image 
generation module 46 so that images of the patient can be 
created and displayed (S19). 

10 Specifically when a wire mesh model is received by the 

image generation model 46 the image generation module 46 
checks with the control module 44 for a current display 
status then proceeds to generate as image of the patient 
8 based upon the current display status. 

One possible display status in this embodiment is a 
status for generating a texture rendered image utilising 
the texture data obtained from the texture cameras 20 of 
the camera rigs 1, 2, 3. In such a configuration the 

20 image generation module 46 causes the frame grabber 5 to 

obtain image data from the texture cameras 20 of the 
common rigs 1,2,3. The image generation module 46 then 
proceeds to texture render the generated three 
dimensional model received from processing module 42 

25 .utilising implicit texture render coordinates from the 
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images received the texture cameras 20 of the three 
camera rigs 1, 2, 3. Thus in this way an image of a 
patient 8 as currently viewed can be shown and displayed 
on the screen 11. 

5 

Alternatively in another configuration, the image 
generation module 46 might be arranged to generate a 
texture rendered image of the wire mesh model received 
from the processing module 42 based upon the relative 

10 differences of the positioning the surfaces of that wire 

mesh model relative to a previously stored wire mesh 
model of the patient 8 stored in the data store 47. Thus 
for example where the surface of the wire mesh model 
received from the processing unit 42 was above the 

15 corresponding portion of a surface of a stored model the 

surface of the model might be rendered red and where it 
was below the stored surface the surface might be 
rendered blue. Thus the image generated by the image 
generation module 46 would provide visual feedback to the 

20 operator to the relative positioning of the patient 8 

during this and a simulation session. 

Alternatively instead of a simple colour rendering of two 
colours, the extent to which a portion of a patient is 
25 above or below the stored surface might be indicated by 
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the relative variation in colouring ranging from a dark 
colour through a light colour where the hue of the 
rendering indicates the relative depth of the patient 
■ relative to the stored data. In such configurations 
5 texture render data for the model would be calculated 

based upon a comparison of the received wire mesh model 
and the stored model obtained from the data store 47. 

After an image has been displayed the next frame of image 
10 data is then processed and further images are generated. 

In this way a series of images are presented to an 
operator providing real time visual feedback of the 
position of the patient. 

15 Second Embodiment 

After an image has been displayed the next frame of image 
data is then processed and further images are generated, 
in this way a series of images are presented to an 
operator providing real time visual feedback of the 

20 position of the patient. 

A second embodiment of the present invention will now be 
described with reference to Figure 10. In the previous 
embodiment, an imaging system was described in which 
25 three camera rigs l, 2, 3 are utilised to obtain image 
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data of a patient 8. In this embodiment, instead of an 
array of cameras 18, 20, 22 on a camera rig, image data 
is obtained utilising a self contained camera unit. The 
rest of the apparatus is unmodified. 

Figure 10 is a schematic block diagram of a camera unit 
in accordance with this embodiment of the present 
invention. The ceunera unit comprises a set of three 
charged coupled devices (CCD's) 70, 71, 72. Each of the 
CCD's 70, 71, 72 in this embodiment con^rise CCD's which 
do not have an asynchronous reset function. Associated 
with each of the CCD's 70, 71, 72 is a timer unit 80, 81, 
82. The timer unit 81 of one of the CCD's 71 is 
connected to a master timing controller 85, so that the 
timer unit 81 can pass a vertical reset signal (VR) to 
the master timing controller 85. This timer unit 81 is 
also connected to the timer units of 80, 82 associated 
with the other two CCD's 70, 72 so that the pixel clock 
signals (PC) for timing the duration for reading data for 
each pixel from the CCD's arrays can be synchronised. 
The master timing controller 85 is also directly 
connected to the two other timing units 80, 82 and is 
also connected to a first and second flash unit 86, 87. 
in this embodiment, the first flash unit 86 is an 
ordinary flash and the second flash unit 87 is arranged 
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to project a speckle pattern for use in matching image 
points . 

Each of the three CCD's is also connected to a analogue 
digital converter 90, 91, 92 to arrange to convert 
signals received from the respective CCD's 70, 71, 72 
into a digital signal which is passed to a frame store 
95. The frame store 95 comprises a latch 97 and a data 
store 98- The frame store 95 and the master timing 
controller 85 are both connected to a local area network 
(IAN) or other high speed interconnect so that the master 
timing controller 85 can receive control instructions 
from a controlling computer 5 and the frame store 95 can 
transmit image data via the IAN to the computer 5 where 
it is processed in a similar way to the data in the first 
embodiment. Additionally, the IAN provides means by which 
activation of three camera units can be synchronised so 
that as in the first embodiment simultaneous images of 
all sides of a patient can be obtained despite some 
portions of a patient being occluded in some viewpoints. 

In use, when image data is to be obtained, the master 
timing controller 70, 71, 72 receives instructions from 
the IAN. These instructions indicate whether the CCD's 
are to be activated simultaneously or asynchronously. 
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If the CCD arrays are to be simultaneously activated an 
activation signal is automatically passed to all of the 
timer units 80^ 81, 82 and is used to activate the first 
flash 86 simultaneously with the shutters of the CCD 
5 arrays- 70-72. After a predetermined exposure period the 

shutters then close. 

In order to synchronise the read out of data from the 
three CCD's , whenever a vertical reset signal is received 

10 by the master timing controller 85 from one of the timer 

units 81 r the signal is automatically passed to the other 
two units. Due to the passing of the vertical reset 
signal (VR) via the master timing controller 85 the 
signals for each unit will not be quite simultaneous in 

15 that a very small difference in timing may occur but this 

difference is not significant. As the vertical reset 
signals (VR) and pixel clock signals (PC) used by the 
three CCD arrays 70-72 are identical, the A/D converters 
90-92 will read and pass to the latch 97 and datastore 98 

20 pixel data which is synchronised and which corresponds to 

images obtained at the same time by all of the CCD 
arrays. Once a complete image has been stored in the 
data store 98 for all three arrays 10-12, the latch then 
releases the images from the data store 98 when 

25 instructed to do so by the master timing controller 85 on 
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the basis of instructions received from the IjRN. 



When asynchronous data is desired, the activation signal 
sent by the master timing controller 85 to the two flash 
units 86, 87 causes the first flash unit 86 to be 
activated shortly before the second flash 87 is 
activated. Simultaneously with the activation of the 
first flash 86, the timer unit 81 of one of the CCD's 71 
causes the shutter on the associated CCD 71 array to 
open. Under control of the master timing controller 85, 
the second flash 87 is activated and the shutters of the 
other CCD's 70, 72 are opened for the duration of the 
second flash. In this embodiment the delay between the 
first and second flashes is made to be equal to a 
predetermined number of pixel clocks for exaiiiple 
corresponding to 8 lines of read out from the CCD's 70- 
72. After a predetermined exposure period the shutters 
for all three CCD's close and image data from the CCD's 
are passed via their respective analogue digital 
converters 90, 91, to the frame store 95. 

As the timer units 80-82 are synchronised, image data 
received via the analogue digital converters 90, 92 will 
also be synchronised. This will mean that image data 
from the two CCD's activated simultaneously 70, 72 will 
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represent image data obtained from the same instant. The 
majority of the image data obtained from the remaining 
CCD will also correspond to the shutter period for the 
two other CCD arrays 70, 72 except that the initial 
5 portion of the data signal transmitted via the analogue 

digital converter 91 will correspond to part of the CCD 
array 71 whilst the shutter was closed. By discarding 
this portion of the data, three images, two slightly 
smaller than the other one but all of which correspond to 
10 near simultaneous exposure of the three CCD arrays 70-72 

can be obtained where two images comprise images of a 
speckle pattern and the third does not. 

When a set of three images have been stored in the frame 
15 store 95 the image data can then be transmitted via the 

LAN to a computer 5 for processing. As this data is 
already in digital form, any suitable means of 
transmission, such as an ethernet can be utilised to 
transmit this data to the computer 5. The costs of 
20 cabling are thereby reduced. Further by providing a 

frame store 95, which is arranged to receive image data 
from the three CCD's 70-72, a means for obtaining 
simultaneous read-out from all three cameras is provided. 



25 



Although in this second embodiment a system has 
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described in which image data is stored in a frame store 
95 and transmitted via a IAN to a computer 5, it will be 
appreciated that a processing system could be 
incorporated within the camera unit so that a 3-D wire 
5 mesh model representing a surface could be transmitted 

rather than having the processing of image data occur 
remotely from the camera unit. 

Further Embodiments and Mod ifications 

10 Although in previous embodiments matching of image 

patches has been described based solely on a transform 
for accounting for different relative orientations of a 
portion of a patient being matched, it will be 
appreciated that additionally, factor such as variation 

15 in relative lighting and differences in gain for 

different cameras or other image sensors, could also be 
taken into account when processing data to match patches. 
The processing to account for such variation would be 
entirely conventional. 



20 
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Although in the first embodiment a system has been 
described in which patches are matched by recalculating 
iterative transforms from the difference between patch 
values at different iterations, it will be appreciated 
that transform values could be determined in a 
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conventional way by updating both the difference matrix 
and the derivative matrix for patches at each iteration. 



Although in the first emboddlment a system identifying the 
isocentre of a treatment apparatus has been described 
utilising a cube having a striped exterior, other systems 
could be utilised. An advantage of the system described 
in the first embodiment is that the identification of 
points corresponding to lines on a flat surface can be 
achieved relatively easily. 

An alternative system could utilise a single flat plate 
on to which as in the previous embodiment laser cross 
hairs identifying the isocentre of a treatment apparatus 
are projected. In the case of a flat plate ^ the 
projection of three lines identifying three planes 
meeting at the point corresponding to the isocentre would 
cause three lines defining a triangle to be seen when 
those lines were projected onto a flat plate held 
obliquely close to the isoceixtre. The positions of three 
corners of the triangle could then be identified from the 
images obtained by the camera rigs. The ortho centre of 
the triangle appearing in the images can then be 
calculated. This is necessarily a perpendicular 
projection of the isocentre onto the plane where the 
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plate was held. The distance of the plate from the 
isocentre can then be calculated utilising the determined 
distances between the points corresponding to the corners 
of the triangle. 

5 

Although in the first embodiment, obliqueness of data is 
utilised to delete overlapping portions of models other 
measures of goodness of match could be used. Thus for 
example the path difference could be used to select 
10 triangles for deletion. 

Further, although in the first embodiment the projected 
triangles are used to identify corresponding portions of 
different models, in other embodiments could use the 
15 identified triangles as a starting point for identifying 

exact matches for corresponding parts of models. 

Additionally, the test for deleting the triangle 
described in the first embodiment could be amended for 
20 example all triangles covered by others which are 

oriented at an angle greater than a threshold could be 
deleted. Alternatively, triangles could be required to 
be both above a threshold and worse than the 
corresponding triangles in order to be deleted. 

25 
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Although in the previous embodiments systems have been 
described in which image processing of stereo images is 
utilised to generate models for positioning a mechanical 
couch, it will be appreciated that the processing system 
described has wider appliceJaility. 

in the radiotherapy field, in addition to generating 
positioning instructions, the data obtained could, be 
utilised in a number of other ways. For example the 
detailed three-dimensional models could be utilised to 
perform contouring to help in the calculation of a 
treatment plan. As the generated contours will be based 
upon the accurate measurements obtained from the camera 
rigs 1, 2, 3. The resulting contours will be highly 
accurate so that an optimum treatment plan can be 
determined . 



Alternatively, the models could be utilised to activate 
the treatment apparatus so that for example the breathing 
cycle of a patient could be monitored and the application 
of radiotherapy applied at the same point during the 
breathing cycle when the patient was closest to a planned 
treatment position. In such a system, the wire mesh 
model data generated would be utilised to control the 
25 treatment apparatus gating so that the synchronisation of 
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treatment and the breathing cycle could be achieved. 

Further instead of matching the position of the patient 
so that treatment and planning eire achieved with the 
5 patient in the same orientation, the present invention 

could be utilised to determine the differences in 
position of the patient so that diagnostic data such as 
Magnetic Resonance Imaging (MRI) or X-ray CT data from 
the same patient could be aligned with one another even 
10 when the patient was not in exactly position when 

different data types were obtained. 

An additional use of the data identifying a patient's 
position would be in the control of the movement of the 
15 treatment apparatus so that collision between the 

apparatus and any portion of a patient's body could be 
avoided. 

The invention is equally applicable to other applications 
20 where the fast generation of a three-dimensional model of 

an individual from camera images is desirable. In view 
of the speed of which models can be generated utilising 
the present invention, the present invention is 
particularly applicable for obtaining three-dimensional 
25 models of a moving individual to identify for example 
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muscular problems in physiology or sports imaging. 

The present invention also has applications outside of 
the medical field for example in body scanning for 
5 clothes or machine vision more generally where three- 

dimensional monitoring of products, for example, 
production lines is desired. The modelling systems could 
also be utilised to obtain data for giving motion capture 
for generation of animation or alternatively for 
10 security applications matching the shape of individuals 

to confirm identity. In security systems the modelling 
unit could be arranged to monitor the content of images 
and model faces of detected individuals whenever a face 
was detected in received images. 

15 

Although the embodiments of the invention described with 
reference to the drawings comprise computer apparatus and 
processes performed in computer apparatus, the invention 
also extends to computer programs, particularly computer 

20 programs on or in a carrier, adapted for putting the 

invention into practice. The program may be in the form 
of source or object code or in any other form suitable 
for use in the implementation of the processes according 
to the invention- The carrier be any entity or device 

25 capable of carrying the program. 
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For example, the carrier may comprise a storage medium, 
such as a ROM, for example a CD ROM or a semiconductor 
ROM, or a magnetic recording medium, for example a floppy 
disc or hard disk. Further, the carrier may be a 
5 transmissible carrier such as an electrical or optical 

signal which may be conveyed via electrical or optical 
cable or by radio or other means. 

When a program is embodied in a signal which may be 
10 conveyed directly by a cable or other device or means, 

the carrier may be constituted by such cable or other 
device or means. 



Alternatively, the carrier may be an integrated circuit 
15 in which the program is embedded, the integrated circuit 

being adapted for performing, or for / use in the 
performance of, the relevant processes. 
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CLAIMS 

1. A method of image processing comprising the steps 
of: 

receiving first and second images representative of 
the same object viewed from a first and a second view 
point ; 

storing for each point in an array of points in said 
first image ^ an estimated transform required to match a 
portion of said first image identified by said point 
corresponding to part of said object to the portion of 
said second image representative of the same part of said 
object; 

identifying an initial seed point within said array 
and adding data identifying said seed point to a queue 
of data identifying points to be processed; and 

sequentially processing each of points identified 
by said queue ^ by: 

adding data to the end of said queue identifying 
points in said array which are adjacent to the point 
identified by data at the head of the queue and for which 
no calculated transfoann has been determined; 

utilising said stored estimated transform for the 
point identified by data at the head of the queue to 
determine a calculated transform for said point to match 
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the portion of said first image identified by said point 
to said corresponding portion of said second image; and 
updating said stored estimated transforms for 
adjacent points in said array to said point for which a 
calculated trans foann is determined utilising said 
calculated transform. 

2. A method in accordance with claim 1 wherein said 
updating of stored estimated transforms comprises the 

steps of: ■ 

determining a first value indicative of the 
correspondence between said portion of said first image 
identified by said point and a portion of said second 
image identified by applying said stored estimated 
trans foanti to said portion of said first image; 

determining a second value indicative of the 
correspondence between said portion of said first image 
identified by said point and a portion of said second 
image identified by applying said calculated transform 
for said adjacent point to said portion of said first 
image ; and 

if said second value is indicative of a closer 
correspondence, replacing said stored estimated transform 
for said point with said calculated transform for said 
adjacent point. 
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3. A method in accordance with claim 2 wherein said 
first and said second images comprise grey scale images 
and said first and said second values comprise calculated 
difference in grey scale values between said portion of 

5 said first image and said identified portion of said 

second image. 

4. A method in accordance with any of claims 1, 2 or 3, 
further comprising storing data identifying in said array 

10 the number of times data, each point is added to said 

queue and only adding data to said queue identifying a 
point if the said point has been added to said queue 
fewer than a predetermined number of times. 

15 5. A method in accordance with any preceding claim 

wherein a stream of video images are received said stream 
of video images comprising pairs of images representative 
of the same object viewed from said first and said second 
view point, wherein said storage of an estimated 

20 transform for matching points in said pairs of images of 

said video stream comprises storing calculated transforms 
for said points in said array for a previous frame of 
images in said video stream. 
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6- A method in accordance with any preceding claim 
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wherein said determination of a calculated transform for 
a point comprises an iterative determination of a 
calculated transform, wherein the initial calculated 
transform for said first iteration corresponds to said 
5 stored estimated transform for said point. 

7. A method in accordance with claim 6 wherein said 
iterative determination comprises determining at each 
iteration a value indicative of the correspondence 

10 between the portion of said first image identified by 

said point and a portion of said second image identified 
by applying said calculated transform for said iteration 
and aborting said calculation if said correspondence is 
greater than a predetermined threshold after a 

15 predetermined number of iterations. 

8. A method in accordance with claims 6 or 7 wherein 
said iterative calculation further comprises at each 
iteration comparing a calculated iterative transform for 

20 said iteration with data identifying one or more 

transforms and aborting said calculation if said 
iterative calculation matches said stored data. 

9. A method in accordance with any of claims 6-8 
25 wherein said iterative determination comprises at each 
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iteration 

determining a difference matrix identifying for each 
point in said portion of said first image identified by 
said point the difference in pixel values for said point 
5 and a corresponding point is said second image identified 

by applying to said points said calculated, transform; 

determining a derivative matrix identifying the rate 
of change of pixel values for said corresponding points 
in said second image; and 
10 utilising said difference matrix and said derivative 

matrix to determine an updated transform. 

10. A method in accordance with claim 9 wherein said 
determining a derivative matrix comprises: 

15 for said first iteration determining said derivative 

matrix utilising said stored estimated transform; and 

for subsequent iterations deteimining an estimated 
derivative matrix utilising the previous derivative 
matrix, and the differences between the previous and 

20 updated transforms and the differences between the 

previous difference matrix and an updated difference 
matrix calculated utilising said updated transform. 
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11. A method in accordance with any preceding claim 
further comprising the step of: 
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when said queue is empty, identifying a further seed 
point within said array for which no calculated transform 
has been determined and adding data identifying said 
further seed point to said queue. 

5 

12. A method in accordance with any preceding claim 
further comprising the steps of: 

determining the relative positions of said first and 

second viewpoints; 
10 illuminating a point in space utilising three 

intersecting planes of light; 

providing an obstruction in the vicinity of said 

point ; 

obtaining first and second images of said 
15 obstruction . illuminated by said intersecting planes of 

light; 

processing said images to determine the relative 
positions of said first and second viewpoints to said 
illuminated point in space; and 
20 utilising said determined transformations and 

positions to calculate the position of points in the 
surface of said object relative to said illuminated point 
in space . 
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13. A method in accordance with claim 12 wherein s 
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obstruction has a striped appearance and said processing 
of images comprises: 

. processing said images of said object to determine 
the positions of points corresponding to illuminated 
portions of said stripes appearing in said images 
relative to said first vi^w point; 

identifying groups of points lying within planes? 
and calculating the position of said point in space from 
the point of intersection of said planes defined by the 



14. A method in accordance with claim 12 wherein said 
obstruction comprises a flat surface and said processing 
of images comprises: 

in said images identifying lines illuminated by said 

planes of light; and 

utilising the positions of the intersections of said 
lines in said images to determine the relative positions 
of said first and second viewpoints and said point in 



15. A method in accordance with any of claims 12-14 
further comprising the step of: 

storing data identifying points on a surface 
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positions of said groups of points. 



20 



space . 
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relative to a point in space; 
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determining a transformation required to match the 
surface identified by said calculated position of points 
with said stored surface. 

5 16. A method in accordance with claim 15 wherein said 

determination comprises: 

determining the projection of said stored points to 
said first view point; 

determining for each of said projected stored 
10 points, the closest points in said array to said 

projected points; and 

calculating said transformation for said surface on 
the basis of the transformations required to match each 
of said stored points to the points in space represented 
15 by the data for the points in the array determined to be 

closest to the respective projections of the stored 
points . 

17. A method in accordance with claim 15 or 16 further 
20 comprising generating movement instructions to cause the 

surfaces of the object in said images to be aligned 
relative to- said illuminated point in the same manner as 
said stored surface is aligned relative to a point in 
space . 

25 



18. A method in accordance in the claim 15 or 16 further 
comprising generating an activation signal when said 
calculated transformation is indicative of a 
trans foiiiiation. of less than a predetermined distance* 

19, A method in accordance with any preceding claim 
further comprising: 

obtaining model data indicative of the surface of 
said object viewed from a third view point; 

utilising said calculated transforms to generate 
model data indicative of the surface of said object 
viewed from said first and second viewpoints; and 

determining portions of said obtained model 
represented by ' said generated model by identifying 
projections of portions of said obtained model which 
project to said first view point and comparing the 
position of portions of said generated model 
corresponding to said projections. 

20 • A method in accordance with claim 19 further 
comprising deleting portions of said obtained model 
determined to be represented by said generated model 
wherein said portions of said obtained model are 
represented by data indicative of surfaces generated from 
data obtained from oblique images- 



21. A method in accordance with claim 20 wherein said 
oblique images are determined utilising said calculated 
transforms for said points in said array in said first 
image corresponding to said portions of said model. 

22. A method in accordance with claims 20 or 21 further 
comprising: 

generating a combined model from said obtained and 
generated models from which portions have been deleted. 

23. Image processing appaxatus comprising: 

a receiver operable to receive first and second 
images representative of the same object viewed from a 
first and a second view point; 

a data store operable to store for each point in an 
array of points in a said first image, an estimated 
transform required to match a portion of said first image 
identified by said point corresponding to part of said 
object to the portion of a said second image 
representative of the same part of said object deceived 
by said receiver; 

a queue store operable to store data identifying 

points to be processed; 

an identification unit operable to identify an 
initial seed point within a said array and adding data 
identifying said seed point to a queue of data 
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identifying points stored in said queue store; and 

processing unit operable sequentially processing 
each of points identified by a queue stored in said queue 
store, by: 

5 adding data to the end of said queue stored in said 

queue store identifying points which are adjacent to the 
point identified by data at the head of the queue and for 
which no calculated transform has been determined; 

utilising said estimated transform stored in said 

10 data store for the point identified by data at the head 

of the queue stored in said queue store to determine a 
calculated transform for said point to match the portion 
of said first image identified by said point to said 
corresponding portion of a said second image received by 

15 said receiver; and 

updating said estimated transforms stored in said 
data store for adjacent points in said array to said 
point for which a calculated transform is determined 
utilising said calculated transform. 

20 

24. An apparatus in accordance with claim 23 wherein 
said processing unit is operable to update estimated 
transforms stored in said data store for points by: 

determining a first value indicative of the 
25 -correspondence between said portion of a said first image 

received by said receiver identified by said point and 



10 
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a portion of a said second image received by said 
receiver identified by applying said estimated transform 
stored for said point by said data store to said portion 
of said first image; 

determining a second value indicative of the 
correspondence between said portion of said first image 
received by said receiver identified by said point and 
a portion of said second image received by said receiver 
identified by applying said calculated transform for said 
adjacent point to said portion of said first image; and 
if said second value is indicative of a closer 
correspondence, replacing said stored estimated transform 
for said point with said calculated transform for said 
adjacent point. 



25. An apparatus in accordance with claim 24 wherein 
said receiver is operable to receive first and said 
second images comprising grey scale images and said 
processing unil^ is operable to calculate as said first 

20 and said second values, calculated difference in grey 

scale values between said portion of said first image and 
said identified portion of said second image. 

26. An apparatus in accordance with any of claims 23, 
25 24 or 25, wherein said data store is further operable to 

store for each point data identifying in said array the 
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number of times data, each point is added to said queue, 
said processing unit being operable to add data to said 
queue identifying a point only if the said point has been 
added to said queue fewer than a predetermined number of 
5 times . 

27. An apparatus in accordance with any of claims 23-26, 
wherein said receiver is operable to receive a stream of 
video images comprising pairs of images representative 

10 of the same object viewed from said first and said second 

view point, wherein said data store is responsive to 
receipt of a new pair of images to store an estdLmated 
transform for matching points in said pairs of images of 
said video stream comprising calculated transforms for 

15 said points in said array for the previous frame of 

images in said video stream. 



28. An apparatus in accordance with any of claims 23-27, 
wherein said processing unit is operable to detemaine a 

20 calculated transform for a point by performing an 

iterative determination of said calculated transform, 
wherein the calculated transform for said first iteration 
corresponds to said estimated transform for said point 
stored in said data store. 

25 

29. An apparatus in accordance with claim 28 wherein 
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said processing unit is operable to detentiine at each 
iteration a value indicative of the correspondence 
between the portion of a said first image identified by 
said point and a portion of a said second image 
5 identified by applying said calculated transform for said 

iteration and aborting said calculation if said 
correspondence is greater than a predetermined threshold 
after a predetermined number of iterations. 

10 30. An apparatus in accordance with claims 28 or 29 

wherein said processing unit is operable to compare at 
each iteration a calculated iterative transform for said 
iteration with data identifying one or more transforms 
and aborting said calculation if said iterative 

15 calculation matches said stored data. 

31. An apparatus in accordance with any of claims 28-30 
wherein said processing unit is operable to determine at 
each iteration: 

20 a difference matrix identifying for each point in 

a said portion of said first image identified by said 
point the difference in pixel values for said point and 
a corresponding point is said second image identified by 
• applying to said points said calculated transform; 

25 a derivative matrix identifying the rate of change 

of pixel values for said corresponding points in said 
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second image; and 

an updated transform determined utilising said 
difference matrix and said derivative matrix. 

5 32. An apparatus in accordance with claim 30 wherein 

said processing unit is operable to determine a 
derivative matrix for a first iteration utilising said 
stored estimated transform; and 

for subsequent iterations determine a derivative 
10 matrix utilising the previous derivative matrix, and the 

differences between the previous and updated transforms 
and the differences between the previous difference 
matrix and an updated difference matrix calculated 
utilising said updated transform. 

33. An apparatus in accordance with any of claims 23-32 
wherein said processing unit is operable when said queue 
is empty to cause said identification unit to identify 
a further seed point within said array for which no 

20 calculated transform has been determined and adding data 

. identifying said further seed point to said queue stored 
in said queue store. 

34. An apparatus in accordance with any of claims 23-33 

25 further comprises: 

a determination unit operable to determine the 
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relative positions of said first and second viewpoints; 

lasers operable to identify a point in space by 
illuminating said point utilising three intersecting 

planes of light; and 
5 an obstruction provided in the vicinity of said 

identified point; wherein received images of said 
obstruction illuminated by said lasers received by said 
receiver are utilised to determine the relative positions 
of said first and second viewpoints to said illuminated 
10 point in space. 

35. An apparatus in accordance with claim 34 wherein 
said obstruction has a striped appearance and said 
. processing unit is operable to: 
15 process said images of said object received by said 

receiver to determine the positions of points 
corresponding to illuminated portions of said stripes 
appearing in said images relative to said first view 
point; and to 

20 identify groups of points lying within planes; and 

calculate the position of said point in space from the 
point of intersection of said planes defined by the 
positions of said groups of points. 

25 36. An apparatus in accordance with claim 34 wherein 

said obstruction comprises a flat surface and said 
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processing unit is operable to: 

identify lines illuminated by said planes of light 
in images received by said receiver; and 

to utilise the positions of the intersections of 
5 said lines in said images to determine the relative 

positions of said first and second viewpoints and said 
point in space. 

37. An apparatus in accordance with any of claims 34-36 

10 further comprising: 

a model store operable to store data identifying 
points on a surface relative to a point in space? and 

a calculation unit operable to determine a 
transformation required to match a surface identified by 
15 said calculated position of points calculated by said 

processing unit utilising said calculated transformations 
and a surface identified by data stored in said model 
Store • 



38. An apparatus in accordance with claim 37 wherein 
said calculation unit is operable to: 

determine the projection of said calculated points 
to a defined view point associating said points 
identified in said stored model in said model store with 
25 points of an array associated with said view point; 

determine for each of said projected calculated 
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points, the closest points in said array to said ' 
projected points; and 

calculating said transformation for said surface on 
the basis of the transformations required to match said 
5 calculated points to the. points in space represented by 

the stored data in said model store, associated with the 
points in said array determined to be closest to the 
respective projections of the calculated points. 

10 39. An apparatus in accordance with claim 37 or 38 

wherein said calculation unit is operable to generate 
movement instructions to cause the surfaces of the object 
in said images to be aligned relative to said illuminated 
point in the same manner as said surface represented by 

15 data stored in said model store is aligned relative to 

a point in space. 

40. An apparatus in accordance with claim 37 or 38 
wherein said calculation unit is operable to generate an 

20 activation signal when said determined transformation is 

indicative of a transformation of less than a 
predetermined distance . 

41. An apparatus in accordance with any of claims 23-40 
25 wherein said apparatus further comprises: 

a merging unit operable to obtain model data 
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indicative of the surface of said object viewed from a 

third view point; and 

determine portions of said obtained model 
represented by a said generated model generated utilising 
said transforms calculated by said processing unit, by 
identifying projections of portions of said obtained 
model which project to said first view point and 
comparing the position of portions of said generated 
model corresponding to said projections. 

42. An apparatus in accordance with claim 41 
further comprising a deletion unit operable to delete 
portions of said obtained model determined by said 
merging unit to be represented by said generated model 
wherein said portions of said model are represented by 
data indicative of surfaces generated from data obtained 
from oblique images. 

43. An apparatus in accordance with claim 42 wherein 
said deletion unit is operable to determine whether data 
is obtained from oblique images utilising said calculated 
transforms f^r said points in said array in said first 
image corresponding to said portions of said model. 

44. An apparatus in accordance with claims 42 or 43 
wherein said merging unit is operable to generate a 
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coiobined model from said obtained and generated models 
from which portions have been deleted. 

45. A method of image processing comprising: 
5 obtaining a stream of video images representative 

of the same object viewed from a first and a second view 
point; 

for each pair of frames representative of said 
object determining for an array of points of an image 
10 from said first view point corresponding points in an 

image from said second view point representative of the 
same part of said object; and 

generating a model of the surface of said object 
utilising the correspondence between points in said pairs 
15 of frames, wherein said determination of corresponding 

points comprises: 

storing for each point in an array of points in said 
first image an estimated transform received to match a 
portion of said first image identified by said point 
20 corresponding to part of said object to the portion of 

said second image representative of the same part of said 
object and utilising said estimated transforms to 
determine said corresponding parts, wherein the initial 
estimated transforms comprise calculated transforms for 
25 matching corresponding points in an earlier pair of image 

frames . 
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46. A method of deterniining corresponding portions of 
models representing the surface of the same object 
comprising: 

obtaining model data representative of the surface 
5 of an object viewed from a first and a second view point; 

determining a projection of model data 
representative of an object viewed from said first object 
to said second view point; 

determining for said projected data whether any 
10 portion of said model data representative of said object 

viewed from said second view point also projects to the 
same portion of said second view point; and 

determining for portions of said models projecting 
to the same portion of said view point, whether the 
15 surfaces represented by said portions of said models are 

close to one another. 

47. A method of aligning two models of surfaces 
comprising : 

20 obtaining model data representative of a first and 

a second model; 

determining for a plurality of points represented 
by said first model, the projection of said points to a 
predefined view point; 
25 for said projected points, determining for said 

second model, the points of said Second model, which 
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project to said predefined view point in the vicinity of 
projected points from said first model; and 

determining an alignment transform for aligning said 
models on the basis of determined transforms for matching 
the points projected from said first model to the closest 
points projected from said second model. 

48. A patient positioning system comprising: 

means for obtaining a stream of images of a patient 

as a mechanical couch from a plurality of viewpoints; 

means for processing frames of images from said 

plurality of viewpoints to generate for each frame a 

model of the surface of said patient; and 

means for determining patient positioning 

instructions on the basis of models generated by said 

means for processing. 



49 . Apparatus for obtaining synchronous and asynchronous 
images of an object from a plurality of viewpoints, said 

20 apparatus comprising: 

a plurality of imaging devices, each imaging device 
comprising an array of charge coupled devices, a shutter 

and a timing unit; 

a control unit operable to vary the timing of the 
opening of said shutters and the read out of image data 
from said imaging devices; and 
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a frame store operable to receive image data from 
said imaging devices; 

wherein the timing unit of one of said imaging 
devices is operable to synchronise the read out of image 
5 data from said charge coupled devices and said control 

pnit is operable to initiate the opening of said shutters 
such that image data obtained from said arrays of charge 
coupled devices corresponds to images obtained whilst 
said all of said shutters are open or to delay the 
10 opening of said shutters of at least one imaging device 

relative to the others wherein the read out of data is 
synchronised with the earliest opening of said shutters. 

50. Apparatus in accordance with claim 49, wherein said 
15 control unit further comprises recognition means for 

identifying the presence of an object in image data 
obtained from said arrays of charge coupled devices 
wherein said control unit is operable to initiate 
asynchronous image capture when the present of an object 
20 is detected. 

51. Modelling apparatus comprising: 

apparatus for obtaining image data in accordance 

with claim 49; and 
25 image processing apparatus in accordance with any 

of claims 23 to 44. 
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52. A data carrier storing computer implementable 
process steps for generating within a programmable 
computer an image processing apparatus in accordance with 
any of claims 23 to 44. 

53. A data carrier storing computer implementable 
process steps for causing a programmable computer to 
perform an image processing method in accordance with any 
of claims 1 to 22. 

54. A data carrier in accordance with claim 52 or 53, 
comprising a computer disc. 

55. A data carrier in accordance with claim 52 or 53, 
15 comprising an electric signal transferred via the 

Internet . 

56. A computer disc in accordance with claim 54, wherein 
said computer disc comprises an optical, magneto-optical 

20 or magnetic disc. 

57. A method of positioning a patient substantially as 
herein described with reference to the accompanying 
drawings . 
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58. Apparatus for positioning a patient as herein 
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described with reference to the accompanying drawings. 



ABSTRACT 



TMAGE PROCESSING SYSTEM FOR USE 
WITH A PATIENT PQSITIQ l^IMG DEVICE 

Three camera rigs (1, 2, 3) are connected by wiring (4) 
to a computer (5). The computer (5) is also connected 
to a treatment apparatus (6). A mechanical couch (7) is 
provided as part of the treatment apparatus (6) such that 
under the control of the computer (5) the relative 
positions of the mechanical couch (7) and the treatment 
apparatus (6) may be varied. The camera rigs (1, 2, 3) 
obtain video images of a patient (8) lying on the 
mechanical couch (7) the computer (5) processes these 
images to generate a three-dimensional model of the 
surface of the patient which is utilised to position the 
patient relative to the treatment apparatus (6). 
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